tail index
- North America > Canada > Ontario > Toronto (0.14)
- North America > United States > Hawaii > Honolulu County > Honolulu (0.04)
- Europe > Switzerland (0.04)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks (1.00)
- Information Technology > Communications (0.68)
Stick-Breaking Mixture Normalizing Flows with Component-Wise Tail Adaptation for Variational Inference
Han, Seungsu, Hwang, Juyoung, Chang, Won
Normalizing flows with a Gaussian base provide a computationally efficient way to approximate posterior distributions in Bayesian inference, but they often struggle to capture complex posteriors with multimodality and heavy tails. We propose a stick-breaking mixture base with component-wise tail adaptation (StiCTAF) for posterior approximation. The method first learns a flexible mixture base to mitigate the mode-seeking bias of reverse KL divergence through a weighted average of component-wise ELBOs. It then estimates local tail indices of unnormalized densities and finally refines each mixture component using a shared backbone combined with component-specific tail transforms calibrated by the estimated indices. This design enables accurate mode coverage and anisotropic tail modeling while retaining exact density evaluation and stable optimization. Experiments on synthetic posteriors demonstrate improved tail recovery and better coverage of multiple modes compared to benchmark models. We also present a real-data analysis illustrating the practical benefits of our approach for posterior inference.
- Asia > South Korea > Seoul > Seoul (0.04)
- Asia > Middle East > Jordan (0.04)
- North America > United States > Tennessee (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks SUPPLEMENTARY DOCUMENT
In Section S4, the relation between compressibility and the tail index is discussed. Proofs of the main results of the paper are presented in Section S5. Finally, the technical lemmas are proved in Section S6. Here we provide a more detailed explanation for our experimental setting, as well as the results and discussion we omitted from the main paper due to space restrictions. Table 1 includes the number of parameters for each model-dataset combination.
- North America > Canada > Ontario > Toronto (0.14)
- North America > United States > Hawaii > Honolulu County > Honolulu (0.04)
- Europe > Switzerland (0.04)